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(54) Abstract Title 

Method and system for preventiRg unauthorized access to a computer program 

(57) A system and method for preventing a program from being run under a debugger utility program. The 
method is part of a routine which is stored along with a software program on a hard drive (16) of a computer 
system (10). The computer system (10) has a processor (12) for running both the software program (26) and 
the routine (100) and is capable of operating in a debug mode. The routine (100) prevents unauthorized access 
to the software program (26), such as when the processor (12) is running in the debug mode. When the 
processor (12) is running the software program (26), the program (26) can initiate execution of the routine 
(100). Once rnitiated, the routine (100) checks a certain register of the processor to determine if it is operating 
in the debug mode and if so, stops the processor (121) from continuing to run the software program. 
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METHOD AND SYSTEM FOR PREVENTING 
UNAUTHORIZED ACCESS TO A COMPUTER PROGRAM 



The present disclosure relates frenerally to computers and computer 
programs, and, more specifically, to a system and method for preventing programs 
irom being run under a debugger utility program. 

A computer program that is stored iiiside a computer often rantains many 
protectable inteUectual properties. Certain ones of these properties are readily 
protectable through copyright and/or patent laws. However, others of these 
properties are protectable as trade secrets, and therefore require the utmost level of 
security to prevent unauthorized use or access. 

Preventing unauthorized use or access of a particular program becomes 
diflScult once the program is attained by a user who is not subject to software Ucenses 
or other forms of protection. For example, diagnostic programs are often instaUed on 
a computer by the computer's manufacturer and contain many trade secrets of the 
manufacturer. TypicaUy. only the executable (.exe) portion of the program is 
instaUed, thereby keeping the source code from the user. Further, the .exe portion of 
the program is often enciypted. For example, a utility program PKLTTE from 
PKWARE. Inc. of Brown Deer, WI, compresses the .exe portion of the program, 
thereby encrypting it in the process. However, this does not prevent a user from 
running the program through a debtigger utiKty program. The debugger utility 
program, makes the computer's processor operate in a suspended execution mode 
which causes the processor to stop execution or jump to another routine at certain, 
predefined intervals. For the sake of example, the processor may be a PENTIUM 
processor, as produced by Intel, Corp. of Santa Clara. CA, and examples of suspended 
execution modes include a debug and a single-step operating mode, all of which are 
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describedingr.aterdetaUmtheIrm:LAECHn^cruBESoFTWAKEDB^I^'s 
MANU^ Vols. 1-3. 1997. pn,vided by Intel Corporation, which is hereby incorporated 
by reference. By using the debugger program, the user can wrong&Uy access the 
program, thereby accessing the trade secrets stored therein. 

Therefore, what is needed is a system and method that would prevent 
unauthori«d access of a program, even through a debugger utility program. 



In response thereto, provided is a system and method for preventing a 
program from being run under a debugger utility program. In one embodiment, a 
n>utine and a software program are stored on a hard drive of a computer system.. 
The computer system has a processor for running both the software program and the 
routineandiscapableofoperatinginasuspendede^cecutionmode. The routme 
prevents unauthorized access to the software program . such as when the proces^MS 
naming in the suspended execution mode under a debugger 

the processor is running the software program, the program can uutxate execution of 
the routine. Once initiated, the n,utine checks a certain predetermined memory 
location to determine if it is operating in the suspended execution mode and ^so. 
stops the processor from continuing to run the software program. 

In one embodiment, the memory location is an interrupt register of the 
processor, which indicates whether any breakpoints, which axe commonly used m 
Lug oper.tingmodes.are being used. The routinechecks the intennapt register to 

determine if it points to any other program or routine. If so. then the processorrs 
U^iynn^ in debugmode and the routine stops the processor from contmumg to 

run the original program. In another embodiment, the memory location is a flag 
register of the processor, which indicates whether other operating modes of the 
processor. In this embodiment, the routine checks the flag register to determme rf U 
indicates a single-step mode of operation for the processor. If so. the routme stops 
the processor &om continuing to run the program. 



3 



An example of the present invention will be desscriheH in ...uu .u. 

accompanying drawings, in which: 

Fig. 1 is a block diagram of a computer, and. 

Fig. 2 is a flow chart of a routine to be run by the computer of Fig. 1 . 

Referring to Fig. 1. the reference numeral 10 designates a computer having 
several components, including at least one processor 12. RAM 14. a hard disk drive 
16. and a floppy disk drive 18. Each component is capable of communication with 
the processor 12. as graphically represented by a gener^ bus 20. In the present 
example, the processor 12 is running DOS operating system code as provided by 
Microsoft Corp. of Redmond. WA. It is understood, however, that the computer 10 
and its illustrated components are merely representative of many different types of 
computers and components and that DOS is merely representative of many different 
types of operating systems, including Windows and Windows 95, also from Microsoft 
Corp. 

The processor 12 includes a plurality of registers, including an mTERRUPT 
register 22 and a FLAG register 24. The names of the registers 22. 24 relate to the 
PENTIUM and PENTIUM U processors, but are only meant to be illustrative of 
similar registers on practically aU models and/or brands of processors. The registers 
22. 24 are also described in the Intel Architecture Software Developer's 
Manual, which has akeady been incorporated by reference and which further 
describes several different processor operating modes, including a normal mode, a 
debug mode, and a single-step mode. 

Stored on the hard drive 16 is a program executable {.exe) ffle 26. The 
program 26 is currently compressed and enoypted using a conventional utility such 
as PKLITE. described in greater detail above. Since the program 26 is enoypted a 
user can not simply read the hard disk 16 to access and decompile the program. 
Conventionally, however, the user would be able to load the program 26 into RAM 14 
and run the program with the processor 12 using a debug utility p^grem (not 
shown). By so doing, the user could learn each line of code in the program 26 and 
decompile the program, thereby attaining unauthorized access to the prx>giam 
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Referring to Fig. 2. a routine 100. which is also stored on the hard disk 16, 
prevents the user from running the program 26 using the debug utility program. 
The program 26 launches the routine 100 during initial operation of the program. 
Therefore, for the sake of «cample. the routine 100 is an initialize routine, it being 
understood, however, that the routine 100 may be run at any time the program 26 is 
being executed. 

At step 102. the routine 100 checks the registers 22. 24. Using the Intel 
PENTIUM processor example discussed above, specifically INTERRUPT register 22 
is a Debug Register 7 and the FLAG register 24 is an EFLAG Register. At step 104. 
the routine 100 determines whether the trap flag (TF) of the EFLAG Register 24 is 
set If TF is set. indicating single-step mode, the routine 100 halts execution of the 
program 26 at step 106. If TF is clear, execution proceeds to step 108. where the 
n,utine 100 determines where the breakpoint flag (INT 3) of U.e INTERRUPT 
Register 22 points. INT3 may point to a routine such as a breakpoint exception 
handler routine, which is often used by a debug program, or it may contain a return 
instruction TRET", which means that no other program or routine is called. If INT 3 
points to any instruction other than IRET. execution proceeds to step 106 described 
above. Otherwise, execution proceeds to step 110 where the routine 100 returns 
confa-ol to the program 26. which may now proceed normally. By checking the 
registers 22, 24, the routine 100 knows whether or not the processor 12 is operating 
in a debug mode, thereby preventing the program 26 ftom being run under a 

debugger utility program. 

Although illustrative embodiments have been shown and described, a latitude 
of modifidition. change and substitution is intended in the foregoing disclosure, and 
in certain instances, some featiires wiU be employed without a corresponding use of 
other features. For example, the routine 100 may be called repeatedly by the 
program 26. thereby further ensuring that the program is not being run by a 
debugger. Furthermore, the program 26 and routine 100 may be stored at different 
locations, additional or alternative registers, flags, or memory devices may be 
checked and additional error handUng routines may be added to Xhe illusti-ative 
embodiment without altering its scope. Accordingly, it is appropriate that the 
appended daims be consumed broadly and in a manner consistent with the scope of 
the invention. 
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Claims 



^ • A computer comprising: 

at least one processor for running a software program, the processor 
capable of operating in a suspended execution mode and having a fir« memory location 
that indicates whether the processor is currently operating in the suspended execution 
mode; 

a storage device accessible by the processor, and 
a routine located in the storage device; 

wherein, when the processor is ranning the software program, the 
program is capable of initiating the routine; and 

wherein, once initiated, the routine checks the first memory location to 
determine if the processor is in the suspended execution mode and if so, stops the 
processor fiom continuing to run the software program. 

2. The computer of claim I, wherein the first memory location is an 
interrupt register of the processor. 

3. The computer of claim 1, wherein the first memoty location is a flag 
register of the processor. 

4. The computer of claim 1 . wherein the processor further includes a second 
memory location, the first memory location being an interrupt register and the second 
memory location being a flag register, and wherein either of the two registers may 
indicate whether the processor is currently operating in the suspended execution mode. 

5. The computer of any one of the preceding claims, wherein the suspended 
execution mode is a debug mode. 

6. The computer of any one of the preceding claims, wherein the suspended 
execution mode is a single-step mode. 
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7 The computer of any one of the preceding claims wherein, if the routine 

determines that the processor is not in the suspended execution mode, the routine returns 
execution to the software program. 

8. The computer of any one of the preceding claims, wherein the storage 
device is a hard disk. 

9. -me computer of any one of claims 1 to 7. wherein the storage device is a 
floppy disk. 

10. n.e computer of any one of claims 1 to 7. wherein the storage device is 
random access memory. 

1 1 A method for preventing a user from running a program on a processor 

operating in a suspended execution mode, the method comprising the steps of. 
checking a first predetermined memory location; 
determining if the first memory location calls another routine; and 
if so, stopping the processor from continuing to run the program. 

12. The method of claim 1 1 further comprising: 
checking a second predetermined memory location; 
determining if the second memory location indicates a suspended 

execution mode of operation for the processor, and 

if so, stopping the processor from continuing to run the program. 

13. The method of claim 1 1 or 12, wherein the first memory location is an 
interrupt register of the processor. 

,4 The method of claim 12, wherein the first memory location is an interrupt 

register of the processor and the second memory location is a flag register of the 

processor. 
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15. The method of any one of claims 1 1 to 14. wherein the suspended 

execution mode of operation is a single-step mode. 

"""^ method of claim 12 or any one of claims 13 to 15 when dependent 
on claim 12, wherein the suspended execution mode of operation is a debug mode. 

' "'^ A '^^v'" preventing a user from running a program on a processor 

operating in debug mode, the device comprising: 

means for checking a first predetermined memory location; 
means for determining if the fust memory location indicates that the 
processor is operating in a suspended execution mode; and 

means responsive to detemining that the first memoiy location indicates 
the suspended execution mode for stopping the processor from continuing to ran the 
program. 

The device of claim 17 fiirther comprising: 

means for checking a second predetermined memoiy location; 

means for determining if the second memory location points to another 

routine; and 

means responsive to determining that the second memory location points 
to another routine for stopping the processor from continuing to run the program. 

<^evice of claim 17 orl8, wherein the first memory location is a flag 
register of the processor. 

The device of claim 1 8, wherein the first memoiy location is a flag 
register of the processor and the second memory location is an interrupt register of the 
processor. 

The device of any one of claims 17 to 21, wherein the suspended 
execution mode of operation is one of either a single-step mode or a debug mode. 
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22 A method of preventing unauthorized access to a co.i.patcr pree,.~T. 

substantially as described with respect to any one of the accompanying drawings. 

23. A computer system substantially as described with respect to any one 

the accompanying drawings. 
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